Addressing Complexity and Intelligence in Systems Dependability Evaluation

Engineering and computing systems are increasingly complex, intelligent, and open adaptive. When it comes to the dependability evaluation of such systems, there are certain challenges posed by the characteristics of “complexity” and “intelligence”. The first aspect of complexity is the dependability modelling of large systems with many interconnected components and dynamic behaviours such as Priority, Sequencing and Repairs. To address this, the thesis proposes a novel hierarchical solution to dynamic fault tree analysis using Semi-Markov Processes. A second aspect of complexity is the environmental conditions that may impact dependability and their modelling. For instance, weather and logistics can influence maintenance actions and hence dependability of an offshore wind farm. The thesis proposes a semi-Markov-based maintenance model called “Butterfly Maintenance Model (BMM)” to model this complexity and accommodate it in dependability evaluation. A third aspect of complexity is the open nature of system of systems like swarms of drones which makes complete design-time dependability analysis infeasible. To address this aspect, the thesis proposes a dynamic dependability evaluation method using Fault Trees and Markov-Models at runtime.The challenge of “intelligence” arises because Machine Learning (ML) components do not exhibit programmed behaviour; their behaviour is learned from data. However, in traditional dependability analysis, systems are assumed to be programmed or designed. When a system has learned from data, then a distributional shift of operational data from training data may cause ML to behave incorrectly, e.g., misclassify objects. To address this, a new approach called SafeML is developed that uses statistical distance measures for monitoring the performance of ML against such distributional shifts. The thesis develops the proposed models, and evaluates them on case studies, highlighting improvements to the state-of-the-art, limitations and future work

A Hierarchical Approach for Dynamic Fault Trees Solution Through Semi-Markov Process

Dynamic fault tree (DFT) is a top-down deductive technique extended to model systems with complex failure behaviors and interactions. In two last decades, different methods have been applied to improve its capabilities, such as computational complexity reduction, modularization, intricate failure distribution, and reconfiguration. This paper uses semi-Markov process (SMP) theorem for DFT solution with the motivation of obviating the model state-explosion, considering nonexponential failure distribution through a hierarchical solution. In addition, in the proposed method, a universal SMP for static and dynamic gates is introduced, which can generalize dynamic behaviors like functional dependencies, sequences, priorities, and spares in a single model. The efficiency of the method regarding precision and competitiveness with commercial tools, repeated events consideration, computational complexity reduction, nonexponential failure distribution consideration, and repairable events in DFT is studied by a number of examples, and the results are then compared to those of the selected existing methods

Online Dynamic Reliability Evaluation of Wind Turbines based on Drone-assisted Monitoring

The offshore wind energy is increasingly becoming an attractive source of energy due to having lower environmental impact. Effective operation and maintenance that ensures the maximum availability of the energy generation process using offshore facilities and minimal production cost are two key factors to improve the competitiveness of this energy source over other traditional sources of energy. Condition monitoring systems are widely used for health management of offshore wind farms to have improved operation and maintenance. Reliability of the wind farms are increasingly being evaluated to aid in the maintenance process and thereby to improve the availability of the farms. However, much of the reliability analysis is performed offline based on statistical data. In this article, we propose a drone-assisted monitoring based method for online reliability evaluation of wind turbines. A blade system of a wind turbine is used as an illustrative example to demonstrate the proposed approach.Comment: A modified version of this work has been published in the 2022 International Conference on Computing, Electronics & Communications Engineering (iCCECE). This work is a draft author versio

Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations

Machine learning is currently undergoing an explosion in capability, popularity, and sophistication. However, one of the major barriers to widespread acceptance of machine learning (ML) is trustworthiness: most ML models operate as black boxes, their inner workings opaque and mysterious, and it can be difficult to trust their conclusions without understanding how those conclusions are reached. Explainability is therefore a key aspect of improving trustworthiness: the ability to better understand, interpret, and anticipate the behaviour of ML models. To this end, we propose SMILE, a new method that builds on previous approaches by making use of statistical distance measures to improve explainability while remaining applicable to a wide range of input data domains

Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations

Machine learning is currently undergoing an explosion in capability, popularity, and sophistication. However, one of the major barriers to widespread acceptance of machine learning (ML) is trustworthiness: most ML models operate as black boxes, their inner workings opaque and mysterious, and it can be difficult to trust their conclusions without understanding how those conclusions are reached. Explainability is therefore a key aspect of improving trustworthiness: the ability to better understand, interpret, and anticipate the behaviour of ML models. To this end, we propose SMILE, a new method that builds on previous approaches by making use of statistical distance measures to improve explainability while remaining applicable to a wide range of input data domains

Keep your Distance: Determining Sampling and Distance Thresholds in Machine Learning Monitoring

Machine Learning~(ML) has provided promising results in recent years across different applications and domains. However, in many cases, qualities such as reliability or even safety need to be ensured. To this end, one important aspect is to determine whether or not ML components are deployed in situations that are appropriate for their application scope. For components whose environments are open and variable, for instance those found in autonomous vehicles, it is therefore important to monitor their operational situation to determine its distance from the ML components' trained scope. If that distance is deemed too great, the application may choose to consider the ML component outcome unreliable and switch to alternatives, e.g. using human operator input instead. SafeML is a model-agnostic approach for performing such monitoring, using distance measures based on statistical testing of the training and operational datasets. Limitations in setting SafeML up properly include the lack of a systematic approach for determining, for a given application, how many operational samples are needed to yield reliable distance information as well as to determine an appropriate distance threshold. In this work, we address these limitations by providing a practical approach and demonstrate its use in a well known traffic sign recognition problem, and on an example using the CARLA open-source automotive simulator

SafeDrones: Real-Time Reliability Evaluation of UAVs using Executable Digital Dependable Identities

The use of Unmanned Arial Vehicles (UAVs) offers many advantages across a variety of applications. However, safety assurance is a key barrier to widespread usage, especially given the unpredictable operational and environmental factors experienced by UAVs, which are hard to capture solely at design-time. This paper proposes a new reliability modeling approach called SafeDrones to help address this issue by enabling runtime reliability and risk assessment of UAVs. It is a prototype instantiation of the Executable Digital Dependable Identity (EDDI) concept, which aims to create a model-based solution for real-time, data-driven dependability assurance for multi-robot systems. By providing real-time reliability estimates, SafeDrones allows UAVs to update their missions accordingly in an adaptive manner

A Markov Process-Based Approach for Reliability Evaluation of the Propulsion System in Multi-rotor Drones

Part 3: Decision SystemsInternational audienceAutonomous multirotor drones as a popular type of Unmanned Aerial Vehicles (UAVs) have a tremendous potential to facilitate activities such as logistics, emergency response, recording video, capturing special events, and traffic management. Despite the potential benefits the possibility of harming people during operation should be considered. This paper focuses on modeling the multirotor drones’ propulsion system with Markov chains. Using the proposed model, both reliability and Mean Time To Failure (MTTF) of the propulsion system are evaluated. This study proposes a fault detection and recovery system based on a Markov Model for mission control of multirotor drones. Concretely, the proposed system aims to reduce potential injuries by increasing safety